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EXECUTIVE  SUMMARY 


This  document  provddes  the  background,  objectives,  methodologies,  and  international  context 
for  conducting  the  reduced  vertical  separation  minima  (RVSM)'  experiment.  It  also  positions 
the  plaimed  simulations  as  strategic  activities  in  the  evaluation  of  RVSM. 

The  experiment  will  use  real-time  simulation  to  evaluate  the  workload  effects  and  feasibility  of 
the  implementation  of  RVSM  in  the  New  York  Air  Route  Traffic  Control  Center  (ARTCC) 
portion  of  tlie  North  Atlantic  Nfinimum  Navigation  Performance  Specification  (MNPS) 
airspace.  The  RVSM  experiment  stems  fi’om  a  conclusion  by  the  North  Atlantic  Systems 
Planning  Group  (NATSPG)  to  cany  out  studies  aimed  at  achieving  early  implementation  of 
RVSM  in  the  North  Atlantic  (NAT)  Re^on.  This  plan  includes  simulations  conducted  in  two 
phases.  These  simulations  will  precede  a  variety  of  other  coordinated  activities  that  will 
culminate  in  the  fuU  implementation  of  RVSM. 

This  document  provides  an  overview  of  the  experiment  plan,  and  more  specirically,  the 
methodologies  to  be  used  in  the  Phase  1  simulation. 


*RVSM  -  1000  ft.  Vertical  Separation  Minima  (VSM)  above  FL  290  up  to  FL  410,  inclusive. 
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1  INTRODUCTION 


The  technical  feasibility  and  cost  benefits  of  establishing  reduced  vertical  separation  minima 
(RV'SM)  in  the  North  Atlantic  (NAT)  Region  have  been  the  subject  of  many  studies  by  affected 
International  Civil  Aviation  Organization  (ICAO)  member  states.  The  results  of  these  studies  have 
led  to  ICAO  planning  for  implementation  of  reduced  minimums  in  January  1998,  with 
trials/verification  scheduled  to  begin  in  January  1996.  In  the  United  States,  the  Federal  Aviation 
Administration  (FAA)  plans  to  conduct  a  series  of  Air  Traffic  Control  (ATC)  simulations  to  assist 
Air  Traffic  organizations  in  identifying  and  defining  requirements  for  the  implementation  of 
RVSM 

The  RVSM  experiment  described  in  this  plan  will  be  conducted  under  the  auspices  of  the 
FAA  National  Simulation  Capability  (NSC)  Program.  The  NSC  will  rely  heavily  on  the  expertise 
of  controllers  from  the  New  York  Air  Route  Traffic  Control  Center  (ARTCC)  oceanic  area  of 
specialization.  NSC  will  also  rely  on  the  expertise  of  the  following  organizations:  Air  Traffic 
Rules  and  Procedures  Service  (ATP- 100),  Air  Traffic  System  Management  (ATM- 100),  Air 
Traffic  Plans  and  Requirements  Service  (ATR-300),  Flight  Standards  (AFS-400),  Operations 
Research  Service  (AOR-20),  Research  and  Development  Service  (ARD-20  and  ARD-100),  and 
Research  Directorate  for  Aviation  Technology  (ACD-300). 

11  BACKGROUND 

In  the  late  1950s  it  was  determined  that  there  was  a  need  to  increase  the  prescribed  vertical 
separation  minimum  (VSM)  of  300  meters  (m)  (1,000  feet  (ft))  due  to  the  inaccuracy  of  pressure 
sensing  barometric  altimeters  as  altitude  increased.  In  1960,  flight  level  (FL)  290  was  selected  as 
the  vertical  limit  for  the  300  m  VSM,  and  a  600  m  (2,000  ft.)  VSM  was  established  for  aircraft 
operating  above  FL  290.  This  vertical  limit  was  chosen  based  on  the  operational  ceiling  of  the 
aircraft  at  that  time.  In  1966,  although  FL  290  was  established  as  the  vertical  changeover  level  on 
a  global  basis,  consideration  was  already  being  given  to  the  application  of  RVSM  above  FL  290 
on  a  regional  basis.  Consequently,  ICAO  provisions  stated  that  RVSM  could  be  applied  under 
specific  conditions  and  within  designated  portions  of  airspace.  In  order  to  support  this  statement, 
it  was  recognized  that  a  thorough  assessment  of  the  risk  associated  with  reducing  the  VSM  would 
need  to  be  accomplished. 

In  1980,  at  its  fourth  meeting,  the  ICAO  Review  of  the  General  Concept  of  Separation  Panel 
(RGCSP)  concluded  that  the  potential  benefits  of  reducing  vertical  separation  above  FL  290  to 
300  m  outweighed  the  cost  and  time  involved  Member  states  were  encouraged  to  conduct  the 
necessary  evaluations.  In  1982,  coordinated  by  the  RGCSP,  studies  were  initiated  to  evaluate 
reducing  the  VSM  above  FL  290.  The  studies  were  carried  out  by  Canada,  Japan,  member  states 
of  EUROCONTROL  (France,  Federal  Republic  of  Germany,  Kingdom  of  the  Netherlands,  and 
United  Kingdom),  the  Union  of  Soviet  Socialist  Republics,  and  the  United  States.  In  December 
1988,  at  the  RGCSP  sixth  meeting  (RGCSP/6),  the  results  were  reviewed. 

Using  a  Target  Level  of  Safety  (TLS)  of  2.5  x  10'^  fatal  accidents  per  aircraft  flight  hour,  it  was 
concluded  that  a  300  m  VSM  above  FL  290  was  technically  feasible.  Today's  height-sensing 
systems  can  be  built,  maintained,  and  operated  such  that  the  expected  performance  is  consistent 
with  safe  implementation  and  use  of  a  300  m  VSM  above  FL  290.  In  reaching  this  conclusion 
about  technical  feasibility,  the  panel  found  that  it  would  be  necessary  to  establish: 
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a.  Air-worthiness  performance  requiremoits  unbodied  in  a  comprehensive  r’inimum  aircraft 
system  performance  specification  (MASPS)  for  all  aircraft  utilizing  the  reduced  separation. 

b.  New  operational  procedures. 

c.  A  comprehensive  means  of  monitoring  the  safe  operation  of  the  system. 

The  RGCSP  identified  the  NAT  Region  as  an  area  where  early  implementation  of  RVSM  was 
possible  because  of  the  traffic  patterns  and  equipment  requirements  of  the  aircraft  population.  On 
this  basis,  and  in  view  of  the  substantial  benefits,  tlw  NATSPG,  at  its  26th  meeting  (Paris,  21 
May-1  June  1990)  agreed  to  carry  out  studies  aimed  at  achieving  early  implementation  of  RVSM 
in  the  NAT  Region.  World-wide  and  regional  provisions  concerning  the  implementation  of 
RVSM  were  finalized  for  applicability  in  November  1992h  Thus,  reduced  vertical  separation  may 
be  implemented  within  \finimum  Navigation  Performance  Specification  (MNPS)  airspace  and/or 
in  other  defined  transition  areas  in  the  ICAO  NAT  Region. 

The  FAA  New  York  ARTCC  persoimel  are  responsible  for  the  provision  of  ATC  services  in  the 
New  York  Oceanic  Control  Area,  part  of  the  ICAO  NAT  Region.  New  Yoric  ARTCC  sectors  71 
and  72  encompass  this  region.  For  RVSM,  their  duties  will  include  transitioning  aircraft  to/fi'om 
domestic  and  non-MNPS/RVSM  airspace  to  RVSM  airspace  (see  figure  1).  Aircraft  en  route 
to/fi'om  RVSM  airspace  traverse  an  area  that  lies  mostly  outside  of  radar  coverage.  A  rignificant 
amount  of  this  area  lies  in  a  portion  of  the  Western  Atlantic  Route  System  (WATRS),  and  thus 
will  be  considered  in  the  RVSM  experiment. 

1.2  PURPOSE. 

The  long  range  air  traffic  forecast  for  the  NAT  Region  estimates  that  air  traffic  will  double  by  the 
year  201(P.  The  reduction  of  vertical  separation  in  the  MNPS  airspace,  NAT  Region,  between 
FL  290  and  FL  410  inclusive,  would  theoretically  accommodate  such  a  projected  increase  in  air 
traffic  and  provide  for  a  more  efficient  use  of  the  available  airspace.  This  enhancement  in  system 
capacity  would  also  result  in  significant  improvements  in  flight  economy. 

It  is  believed  that  the  most  difficult  problems  associated  with  operating  under  RVSM  in  the 
MNPS  airspace  would  be  the  transition  of  aircraft  to  conventional  vertical  separation  minima 
(CVSM)^.  Additionally,  the  procedures  for  transition  may  differ  based  upon  the  geographical 
restrictions. 

There  are  two  geographical  areas  that  have  been  identified  as  potential  transition  areas.  The  first 
transition  area  to  be  evaluated  will  be  witlun  the  MNPS  airspace  (see  figure  2).  Transition  within 
this  area  will  require  all  westbound  aircraft  to  be  separated  by  CVSM  prior  to  leaving  unless  the 
aircraft  are  entering  radar  coverage  in  the  depicted  northern  portion  of  sector  72  or  into  Canadian 
airspace  which  is  either  MNPS  or  radar  coverage. 


^Manual  on  I^^)lenlentation  of  a  300  m  (1000  ft)  VSM  between  FL  290  and  FL  410  illusive  is  ICAO  Doc.  No. 
9574-AN/934,  dated  1992. 

^Agenda  Item  2,  Woridng  Paper  13 1  agenda  presented  at  the  Limited  North  Atlantic  (COM/MET/RAQ  Regional 
Air  Navigation  Meeting  held  in  Cascais,  Portugal,  in  November  1992. 

^CVSM  -  2000  ft  VSM  above  FL  290  up  to  FL  430,  inclusive. 
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HGURE  1.  MNPS/RVSM  AIRSPACE  UNDER  NEW  YORK  ARTCC  CONTROL 
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FIGURE  2.  PHASE  1  TRANSITION  AREA 
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The  second  transition  area  (see  figure  3)  will  have  use  of  radar  coverage  available  for  transitioning 
RVSM-approved  aircraft  from  RVSM  to  CVSM  standard  The  transition  would  occur  when 
RVSM-approved  aircraft  have  entered  the  radar  coverage  area  and  after  two-way  direct  very  high 
frequency  (VHP)  communications  have  been  established  It  must  be  noted  that  those  aircraft  that 
transition  within  Bermuda  radar  coverage  will  be  permitted  to  traverse  a  small  portion  of  the 
WATRS  airspace  while  still  using  RVSM  flight  rules  It  is  envisioned  that  future  experiment 
phases  will  focus  on  other  areas  that  may  be  used  for  transition  such  as  the  WATRS  area 

ATC  procedures  for  the  various  potential  RVSM  transition  areas  within  the  NAT  region  and 
adjacent  ICAO  regions  need  to  be  defined  prior  to  implementation  The  proposed  RVSM 
experiment  will  evaluate  procedures  used  by  controllers  in  transitioning  aircraft  from  RVSM  to 
CVSM 

The  NSC  RVSM  experiment,  and  associated  activities,  are  designed  to  provide  Air  Traffic  service 
organizations,  especially  the  International  Procedures  Branch  (ATP- 140),  with  the  vital  human 
performance  information  needed  to  define  RVSM  implementation  procedures  The 
findings/conclusions  will  be  closely  coordinated  and  shared  with  all  NAT  ATC  provider  states  to 
help  facilitate  the  development  of  a  unified  implementation  plan  An  ICAO-developed  activity 
schedule  ensures  that  all  efforts  are  coordinated  (see  appendix  A) 

I  3  OVERALL  OBJECTIVES 

The  first  objective  of  the  RVSM  experiment  is  to  identify  the  geographical  areas  where  aircraft 
can  be  safely  transitioned  from  RVSM  to  a  CVSM  environment,  and  to  establish  procedures  to 
effect  the  transition 

Secondly,  the  experiment  will  address  changes  in  controller  workload  that  may  be  attributed  to 
increased  flight  operations  and  problems  associated  with  transitioning  RVSM-approved  aircraft  to 
and  from  the  designated  RVSM  area.  Part  of  this  objective  is  to  study  the  impact  on  the 
controller  of  weather-related  or  other  contingencies  that  could  cause  an  aircraft  to  deviate  from  an 
assigned  altitude  and/or  flight  path.  The  study  will  evaluate  the  ability  of  controllers  to  work 
RVSM  traffic  and  to  issue  instructions  to  pilots  in  sufficient  time  to  allow  a  resolution  of 
projected  conflicts.  It  should  be  noted  that  operational  boundaries  for  sectors  change  from  day  to 
day  in  the  operational  environment  to  allow  for  equitable  distribution  of  workload 

A  separate  experiment  in  Fiscal  Year  1994  will  evaluate  and  compare  the  current  New  York 
ARTCC  oceanic  ATC  capabilities  with  the  planned  future  ATC/aircraft  capabilities.  The 
projected  system  capabilities  will  include  expected  availability  of  Automatic  Dependent 
Surveillance  (ADS),  Global  Navigation  Satellite  System  (GNSS),  Enhanced  Situation  Displays 
(ESDs),  enhanced  automation,  etc.  The  projected  system  capabilities  will  be  based  on  those 
envisioned  for  1995  that  are  listed  in  the  Oceanic  Concept  of  Operations  FAA  document.  This 
document  was  developed  over  the  past  two  years  by  the  FAA  Oceanic  System  Requirements 
Team  (OSRT) 
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These  experiineats  represent  e  critical  stq>  by  Air  Traffic  Service  in  assessing  current  and 
projected  New  Yoik  ARTCC  oceanic  ATC  system  capabilities.  They  will  provide  the  basdine  for 
the  identification  of  additional  system  requirements,  if  needed,  and  provide  important  information 
for  the  devdopment  of  an  FAA  RVSM  implementation  plan. 

1 A  APPROACH. 

The  experiment  will  incorporate  real-time  ATC  Emulations  designed  to  evaluate  woridoad  when 
controUers  apply  separation  and  other  ATC  services  in  designated  oceanic  transition  airspace 
areas.  Currently,  most  of  the  candidate  transition  oceanic  airspace  areas  are  non-rada<’  areas. 
However,  one  or  more  of  the  designated  transition  areas  may  fidl  into  airspace  wnere  radar 
coverage  is  available. 

T  .  approach  of  the  RVSM  experiment  will  adhere  to  the  internationally  agreed  upon  guidelines 
as  a  bads  for  the  devdopment  and  definition  of  simulation  scenarios.  These  guidelines  are: 

a.  RVSM  will  be  effected  coincident  with  MNPS  airspace  and  in  defined  transition  areas. 

b.  The  transition  to/fiom  reduced  CVSM  should  be  effected  in  trandtion  areas. 

c.  The  trandtion  areas  will  be; 

1.  Defined  as  Class  A  airspace  wiUun  vdiich  aircraft  proceeding  to/fi*om  MNPS  airquice 
will  be  authorized  to  trandtion  to/fiom  1,000  ft  VSM. 

2.  Contained  within  horizontal  limits  determined  by  provider  states,  dther  individually  or 
in  consultafion. 

3.  Adjacent  to,  oveii^ping  with,  or  contained  within,  MNPS  airspace. 

4.  Within  radar  coverage  using  direct  controUer/pilot  communications  wherever  practical. 

5.  Contained  within  vertical  limits  of  FL  290  to  FL  410,  includve. 

d.  When  operating  within  trandtion  areas,  RVSM  may  be  applied  between  those  aircraft 
approved  for  such  operations  when  trandtioning  to/fi’om  MNPS  airspace.^ 

The  initial  experiment  ^11  be  accomplished  in  two  phases  (Emulations)  that  investigate  and 
measure  the  effects  of  RVSM  implementation.  Each  phase  wiU  require  the  development  and 
simulation  of  several  different  air  traffic  scenarios.  The  phases  are: 

a.  Phase  1.  Study  trandtion  of  westboimd  aircraft  from  RVSM  to  CVSM  before  lea>dng 
RVSM/MNPS  airspace  (see  figure  2)  with  the  reception  of  aircraft  that  enter  radar  coverage 
adjacent  to  RVSM/MNPS  airspace  or  Canadian  airspace  which  is  either  MNPS  or  radar  coverage. 


^Agenda  Item  2,  Working  Filler  131  presented  at  the  Limited  North  Atlantic  (COM/MET/RAQ  Rqpcmal  Air 
Navigation  Meeting  held  in  Cascais,  Portugal,  in  November  1992. 
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b.  Phase  2.  Investigate  the  acceptability  of  the  use  of  the  radar  cove,  age  available  for 
transitioning  RVSM-approved  aircraft  from  RVSM  to  CVSM  standard.  The  transition  vi/ould 
occur  when  RVSM-approved  aircraft  have  entered  the  radar  coverage  area  and  after  two-way 
direct  VHP  communications  have  been  established  (see  figure  3) 

Upon  the  successful  completion  of  these  two  phases,  additional  testing  of  other  potential 
transition  areas  may  be  investigated  For  example,  aircraft  traversing  through  the  WATRS  area 
would  be  allowed  to  continue  using  RVSM  rules  and  transition  would  occur  within  radar 
coverage  provided  by  surrounding  domestic  centers  (i.e.,  Miami,  Jacksonville,  Washington,  and 
Boston). 

These  experiments  will  incorporate  the  study  of  aspects  alluded  to  in  the  previously  mentioned 
objectives.  They  are  expected  to  provide  valuable  information  in  assessing  RVSM 
implementation  strategy  and  ATC  system  requirements  for  other  NAT  ATC  providers  Other 
NAT  Region  ATC  provider  states  affected  by  RVSM  implementation  are  also  planning  similar 
experiments. 

2  PHASE  1.  SIMULATION  DESIGN 

This  section  defines  the  objective,  approach,  simulation  methodology,  and  analysis  planned  for  the 
Phase  1  simulation  of  the  RVSM  Experiment. 

2  I  SIMULATION  DESIGN  OBJECTIVE 

The  objective  of  this  simulation  is  to  measure  controller  workload  attributed  to  changes  in 
standard  operating  procedures.  This  phase  will  do  the  following: 

a.  Measure  changes  in  controller  workload  levels  as  impacted  by  RVSM  operations. 

b.  Identify  operational  issues  associated  with  RVSM  operations 

c.  Identify  any  operational  difficulties  associated  with  controllers'  ability  to  transition 
aircraft  from  RVSM  to  CVSM  within  MNPS  airspace. 

d.  Identify  other  issues  related  to  reverting  from  RVSM  rules  to  CVSM  rules. 

2  2  SIMULATION  DESIGN  APPROACH 

Each  simulation  parameter  (scenario,  protocol,  and  instrumentation)  has  been  designed  to  enable 
valid  and  informative  comparisons  between  current  CVSM  operations  and  RVSM  conditions. 
Toward  this  purpose,  overall  traffic  load  will  be  held  constant  for  each  simulation  run,  with  only 
the  separation  rules  and  constraints  being  varied  under  each  condition. 

2  3  SIMULATION  DESIGN  METHODOLOGY 

The  simulation  will  be  carried  out  in  the  Oceanic  Development  Facility  (ODF)  located  at  the  FAA 
Technical  Center.  The  immediate  physical  environment  (as  shown  in  figure  4)  realistically 
simulates  the  New  York  ARTCC,  including  the  available  equipment  and  communication 
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interfaces.  All  flights  will  be  generated  by  the  ODF.  The  supporting  Aeronautical  Radio 
Incorporated  (ARJNC)  communications  will  also  be  simulated.  The  ATC  environment  will  be 
fully  instrumented  (video  recording,  audio  recording,  and  computer  recording)  to  support  the 
collection  of  controller  performance  data. 

2.3.1  ODF  Laboratory  Configuration. 

The  ODF  was  developed  to  provide  a  complete  oceanic  ATC  environment  to  support  realistic 
testing  and  evaluation  of  oceanic  ATC  equipment,  interfaces,  and  procedures  The  configuration 
of  the  ODF  laboratory  will  include  two  complete  oceanic  control  positions,  each  consisting  of  an 
M 1  console,  including  strip  bays,  integrated  Flight  Data  Input/Output  (FDIO)  equipment,  voice 
communication  equipment,  flight  strip  printers,  emulated  ARINC  printers,  and  overhead  sector 
charts.  It  should  be  noted  that  only  one  Oceanic  Display  and  Planning  System  (ODAPS)  Plan 
View  Display  (PVD)  will  be  used  in  the  simulation.  This  is  in  accordance  with  normal  operating 
procedures  for  New  York  ARTCC  oceanic  control. 

External  interfaces  will  be  exercised  as  they  would  be  in  the  field.  Controllers  involved  in  the 
simulation  will  also  be  exposed  to  a  realistic  control  environment.  Voice  communications  to 
ARJNC  and  adjacent  facilities  will  be  simulated.  Simulated  aircraft  will  react  dynamically  to 
controller-issued  clearances. 

Key  components  of  the  ODF  are  described  in  the  following  paragraphs. 

The  scenario  generator  accepts  scenario  data  provided  by  the  lab  user  taken  from  actual 
operations  (System  Analysis  and  Recording  (SAR)  tapes).  The  test  analyst  has  the  capability  to 
list  planned  event  data  interleaved  with  aircraft  way  point  crossings  in  chronological  order, 
associating  an  expected  simulation  time  with  each  element  in  the  list 

The  target  generator  provides  position  reports  that  represent  the  trajectory  of  simulated  aircraft 
specified  in  the  scenario.  The  position  reports  reflect: 

a.  The  effects  of  simulated  forecast  winds. 

b.  Controller  clearances. 

c.  On-line  adjustments  to  the  route  made  by  the  remote  operator 

The  remote  operator  engages  in  voice  communications  with  controllers,  effectively  simulating  an 
ARINC  operator  relaying  voice  communications  to  and  from  the  pilot  of  a  controlled  aircraft  in 
the  controller’s  sector,  a  controller  in  an  adjacent  Flight  Information  Region  (FIR)  transferring 
control  and  accepting  aircraft  or  performing  other  coordination  functions,  and  a  controller  at 
another  sector  or  ARTCC. 

The  remote  operator  will  enter  controller  clearances  to  the  ODF.  The  remote  operator  will  also 
accept  an  agreement  made  with  another  simulated  ATC  entity  (such  as  an  adjacent  FIR)  and  enter 
it  into  the  ODF.  The  ODF  uses  these  entries  to  record  and  modify  flight  data. 
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2  3  2  ODAPS 


Thii  comonent  provides  oceanic  flight  data  processing  and  oceanic  display  capabilities  for 
selected  domestic  ARTCCs  that  have  oceanic  control  responsibilities  ODAPS  performs  flight 
data  processing  for  oceanic  flights  in  three  sectors  of  the  New  York  ARTCCs  area  of 
responsibility.  ODAPS  processes  flight  plan  data  and  related  messages  in  conjunction  with  stored 
adaptation  data  to  produce  outputs  to  be  transmitted  via  FDIO  control  units  to  FDIO  equipment 
located  at  the  oceanic  sector  positions  The  FDIO  equipment  will  use  the  data  output  by  ODAPS 
to  print  flight  strips  and  other  messages  essential  to  oceanic  ATC  at  the  appropriate  sector 
positions.  ODAPS  also  provides  the  oceanic  controllers  with  a  graphical  representation  of  the 
flight  plan  extrapolated  position  of  all  aircraft  under  their  control. 

2  3.3  Simulation  Participants. 

Four  controllers  from  New  York  ARTCC.  all  sector  certified,  will  participate  in  Phase  1.  These 
controllers  will  rotate  the  staffing  ofthe  two  "D"  and  one  ODAPS  positions.  Two  ARfNC  radio 
operators  will  be  used  to  staff  the  remote  operator  position.  Two  technical  observers  from  New 
York  ARTCC  will  be  trained  to  observe  and  record  subjective  performance  data  on  the 
controllers  working  the  sectors  Additional  support  staff  will  be  provided  by  ACD-330.  ACD- 
340,  ACD-350,  and  ACN-600  of  the  FAA  Technical  Center. 

2  3.4  Scenario  Description. 

New  York  oceanic  environment  data  were  obtained  from  an  Adaptation  Controlled  Environment 
Subsystem  (ACES)  tape  from  March  18,  1993.  Flight  plans,  including  a  mix  of  aircraft  types  and 
equipment  (domestic  versus  foreign  carrier,  military  versus  commercial  carrier)  were  created  from 
a  Data  Analysis  and  Reduction  Tool  (DART)  run  of  the  SAR  tape  dated  March  18,  1993.  The 
scenario  represents  a  westbound  traffic  flow  on  the  five  published  NAT  tracks  and  on  random 
routes  from  1400Z  -  1800Z.  The  airspace  the  controller  participants  will  be  responsible  for 
includes  sectors  7 1  and  72  of  the  New  York  ARTCC  (see  figure  2).  The  flight  strips  will  be 
printed  and  placed  in  the  flight  strip  bay  at  appropriate  times  by  support  personnel.  Planned 
events  based  on  March  18  (such  as  aircraft  reports  of  turbulence,  altitude  change  requests,  routine 
position  reports,  emergencies,  etc.)  will  be  scripted  and  used  in  the  appropriate  simulation 
conditions.  These  events  will,  in  most  cases,  be  introduced  via  the  remote  operator  position  at 
pre-scheduled  times. 

2  3.5  Performance  Measures. 

To  date,  an  insufficient  amoun:  of  work  has  been  conducted  on  the  assessment  of  controller 
workload  in  the  oceanic  control  environment.  Therefore,  it  will  be  necessary  to  determine  which 
controller-performance  measures  in  the  oceanic  environment  are  suitable  as  indicators  of 
controller  workload.  Questionnaires  will  be  provided  to  controllers  and  simulation  results  will  be 
reviewed  to  identify  critical  indicators  of  controller  workload.  Sample  forms/questionnaires  are 
provided  in  appendix  B. 

2  3  5  1  Objective  Measures. 

Several  aspects  of  oceanic  controller  performance  are  being  considered  as  indicators  of  controller 
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workload,  including,  but  not  limited  to 


a  Delays. 

b.  Response  to  user  requests, 
c  Rerouting. 

d.  Restrictions 

e.  Operational  errors 

f.  Timeliness  of  controller-initiated  instructions 

g.  Total  number  of  altitude  changes 

h  Number  of  altitude  changes  experienced  per  aircraft 
i  Amount  of  coordination/communications 

j.  Ability  to  transition  aircraft  from  RVSM  to  CVSM  within  the  MNPS  area. 

These  performance  measures  will  be  assessed  comparatively  and  will  be  expressed  as  a 
"percentage"  change  from  the  baseline  CVSM  simulation  runs. 

2.3.5  2  Subjective  Measures. 

In  addition  to  the  objective  performance  measures,  the  following  subjective  measures  will  be 
collected  to  assess  levels  of  controller  workload; 

a.  "Real-time"  ratings  of  workload  obtained  by  having  the  technical  observer  prompt  a  verbal 
report  from  all  controllers  at  regular  time  intervals  (approximately  every  1 5  minutes)  throughout 
each  simulation  run.  This  procedure,  called  the  Air  Traffic  Workload  Input  Technique  (ATWIT), 
is  an  FAA-validated  technique  used  for  the  assessment  of  controller  workload  on  a  continual 
basis. 

b.  Controller  responses  to  technical  observer-administered  structured  interviews  and 
questionnaires  conducted  after  each  baseline  CVSM  and  each  RVSM  simulation  run. 

c.  Real-time  observations  of  critical  controller  actions  recorded  by  trained  technical 
observers  throughout  each  simulation  run. 

2  3  6  Data  Collection. 

Comparisons  between  the  workload  reported  at  baseline  CVSM  and  at  RVSM  conditions  1,  2  and 

3  (as  defined  in  section  3.2)  will  be  based  on  data  collected  via: 

a.  Automated  recording  of  ODAPS  via  SAR  tapes. 

b.  Observations  of  critical  controller  actions  confirmed  by  analyzing  audio-video  records 
of  the  simulation  runs. 

c.  Analysis  of  flight  progress  strips  used  during  simulation  runs. 

d.  Observation  or  frequency  counts  of  critical  controller  actions  made  in  real-time  by 
technical  observers  with  "quick"  data  collection  forms. 
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e.  Controller  responses  to  structured  interviews  and  questionnaires  conducted  after  each 
baseline  CVSM  and  each  RVSM  simulation  run. 

f.  Other  available  computer-recorded  performance  data. 

2  4  ANALYSIS  APPROACH 

Objective  and  subjective  data  analysis  will  identity  controller  workload  issues.  Descriptive 
statistics  will  be  used  to  organize  and  summarize  simulation  metrics.  In  addition,  inferential 
statistics  will  be  used  as  necessary  to  address  other  specific  questions  of  interest  that  may  arise. 

2.4. 1  Data  Analysis. 

The  data  will  be  collected  from  simulation  runs  and  will  be  analyzed  in  an  effort  to  detect  any 
change  in  controller  performance  The  analysis  period  between  simulations  serves  two  purposes: 

( I )  the  controller  performance  measures  can  be  established,  and  subsequent  instances  of  these 
measured  conditions  can  be  used  for  comparison;  (2)  the  results  of  data  may  indicate  additional 
controller  performance  measures  that  may  prove  useful  as  indices  of  controller  workload  in  the 
oceanic  environment.  The  overall  purpose  of  the  data  analysis  for  Phase  1  is  to  address  the  issue 
of  whether  workload  changes  occurred.  The  analysis  will  also  summarize  any  comments  and 
observations  made  by  controllers  and  technical  observers  that  address  procedural  requirements 
regarding  operations. 

2.4  2  Analysis  Procedures 

In  addition  to  reporting  standard  descriptive  statistics  (e  g  ,  mean,  frequency,  standard  deviation, 
etc.),  the  analysis  may  be  composed  of  a  number  of  inferential  statistics  These  inferential 
statistics  may  include  Analysis  of  Variance  (ANOVA),  protected  t-tests,  tests  for  normality  and 
homogeneity  of  variance,  regression  analysis,  correlation,  and  non-parametric  methods.  The 
purpose  of  performing  any  inferential  statistical  analysis  would  be  to  attempt  to  identify  patterns 
of  performance  or  controller  behavior  that  would  be  useful  in  addressing  procedural  requirements 
regarding  RVSM  operations. 

3  PHASE  I  SIMULATION  PROCEDURES 

The  following  sections  describe  the  procedures  for  the  Phase  I  simulation. 

3  1  STAFFING  and  TRAINING  REQUIREMENTS. 

The  following  section  describes  the  staffing  and  training  requirements  for  the  Phase  I  simulation. 
The  staffing  section  details  the  personnel  required  from  both  the  New  York  ARTCC  and  the  FAA 
Technical  Center.  The  training  section  describes  the  training  for  the  controllers  and  the  technical 
observers. 
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3  11  Staffing 


The  simulation  will  be  staffed  by  four  air  traftk  controllers,  all  currently  certified  within  the  past 
two  years  on  sector  71  and  72,  from  New  York  ARTCC.  Additionally,  the  New  York  ARTCC 
will  provide  two  training  officers  to  participate  as  technical  observers  during  the  simulation  Two 
ARJNC  radio  operators  will  also  participate  The  FAA  Technical  Center  will  provide  the 
technical  support  staff  (human  factors  engineers,  statistical  analysts,  and  ODF  technical  support) 
to  conduct  the  simulation  and  analyze  the  results 

312  Training 

The  following  sections  describe  the  training  that  will  be  necessary  for  select  categories  of 
technical  staff 

3  12  1  Controller  Participants 

New  York  ARTCC  controllers  will  be  given  several  hours  of  practice  using  the  FAA  Technical 
Center's  ODF  prior  to  baseline  data  collection,  to  familiarize  themselves  with  the  configuration 
and  system  performance.  Although  every  attempt  is  being  made  to  duplicate  the  operational 
environment  of  the  New  York  ARTCC  oceanic  work  area,  the  FAA  Technical  Center's  ODF  may, 
in  some  ways,  differ  slightly  in  system  performance,  interface  responsiveness,  and  workspace 
configuration.  Familiarization  training  will  include  limited  simulation  runs  specifically  scripted  to 
exemplify  the  configurations  of  the  FAA  Technical  Center  ODF  so  that  the  controllers  can 
become  accustomed  to  any  differences  from  the  New  York  ARTCC  oceanic  control  facility.  The 
controllers  will  be  trained  and  directed  on  special  procedures  for  the  RVSM  environment. 

3.12  2  Technical  Observers. 

The  training  of  technical  observers  will  occur  before  and  during  the  practice  simulation  runs 
conducted  prior  to  baseline  data  collection.  During  this  training,  the  observers  will  become 
familiar  with  their  roles  in  the  simulation  runs.  Since  practice  simulation  runs  are  specifically 
scripted  to  familiarize  controllers  with  the  FAA  Technical  Center  ODF,  the  observers  will  have  an 
opportunity  to  practice  recording  controller  performance  in  the  actual  experimentation  system 
configuration. 

This  training  procedure  will  familiarize  the  observers  with  the  critical  controller  actions, 
behaviors,  and  occurrences  that  will  be  used  in  the  assessment  of  controller  workload.  Further, 
observers  will  be  trained  to  record  this  information  on  the  "quick"  reporting  form  that  they  will 
have  for  each  controller  within  each  simulation  run;  the  purpose  of  this  instruction  is  to  keep 
errors  in  the  hand-recording  of  data  to  a  minimum  and  to  optimize  rating  reliability. 

3  2  SIMULATION  DESIGN  PROCEDURES 

The  following  sections  provide  a  description  of  the  simulation  procedures  and  the  subject 
scheduling. 
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3  2  1  General  Procedures 

Phase  1  will  be  composed  of  two  part--',  with  each  part  containing  two  expenmental  conditions 
There  will  be  two  simulation  runs  per  experimental  condition  for  a  total  of  eight  simulation  runs 
during  Phase  1.  All  simulation  runs  will  use  the  New  York  ARTCC  oceanic  traffic  (aircraft  types, 
numbers,  and  tracks),  which  occurred  on  March  18,  1993,  as  the  basis  to  construct  each  scenario 
March  1 8  has  been  chosen  because  it  represents  a  full  capacity,  five-track  traffic  flow  day.  All 
simulation  runs  will  occur  in  a  two-sector  configuration  (sector  7 1  and  sector  72)  with  each 
having  a  "D"  and  sharing  a  single  ODAPS  controller  position.  Each  simulation  run  will  last 
approximately  6.5  hours  Controllers  will  rotate  through  the  controller  positions  so  that  each 
controller  will  break  for  30  minutes  after  working  a  position  for  approximately  1 1/2  hours  During 
the  position  exchange,  no  pause  or  break  in  the  simulation  will  occur,  replicating  a  realistic  work 
environment. 

There  will  be  two  scenarios  developed  for  each  experimental  condition  The  two  scenarios  are 
identical  to  each  other  with  the  exception  that  planned  events,  such  as  change  in  altitude  request, 
are  made  by  different  aircraft  This  is  done  to  minimize  the  opportunity  for  controllers  to  "learn" 
the  script  of  the  scenarios  due  to  repeated  exposure  The  two  different  scenarios  will  be  run  on 
two  consecutive  simulation  days. 

Two  AJUNC  radio  operators  will  be  required  to  assist  in  the  remote  operator  position,  which  will 
simulate  contacts  with  ARJNC,  other  domestic  and  oceanic  controllers,  and  other  FlRs  In 
addition,  a  technical  observer  will  record  specific  activities  at  each  sector. 

The  comparison  between  CVSM  baseline  operations  and  RVSM  operations  will  be  of  prime 
importance  in  providing  information  regarding  controller  workload  levels  By  usi.ig  the  same 
group  of  oceanic  controllers  for  both  the  CVSM  baseline  operation  and  the  RVSM  operation,  it 
will  be  possible  to  account  for  changes  in  controller  strategies  for  working  traffic.  Hence,  major 
differences  in  controller  performance  can  be  attributed  to  the  impact  that  the  different  operations 
have  on  controller  workload. 

3.2.2  Baseline  CVSM/Basic  RVSM  Experimental  Conditions/Runs . 

Part  1  (four  days  in  duration)  consists  of  two  experimental  conditions  (see  table  1).  The  first  is  a 
two  day  CVSM  baseline,  and  second  is  a  two  day  RVSM  basic  (altitude  compression  only).  As 
previously  stated,  ihe  baseline  CVSM  and  the  basic  RVSM  experimental  conditions  will  be 
conducted  for  two  days  each.  The  baseline  CVSM  scenarios  will  use  the  traffic  load  of  March  18. 
However,  the  basic  RVSM  scenarios  will  use  the  same  traffic  as  CVSM,  except  that  some  aircraft 
will  be  reassigned  even  altitudes  representing  the  application  of  RVSM  flight  rules. 

3  2.3  RVSM-Minimum  Time  Track  (MTTVRVSM-Clear  Air  Turbulence  (CAT)  Experimental 
Conditions/Runs. 

Part  2  (four  days  in  duration)  consists  of  two  experimental  conditions  (see  table  1).  The  first  is  a 
two  day  RVSM  (RVSM-MTT)  with  aircraft  moved  closer  to  the  MTT,  and  second  is  a  two  day 
RVSM  (RVSM-CAT)  with  reports  of  greater  than  moderate  CAT.  On  the  first  two  days. 
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Table  1  -  Controller  Assicnments:  NSC  Phase  1  Simulation 


All  conditions  Day  1  &  Day  2 
4  Controllers  A.  B.  C.  D 


RVSM-MTT  will  consist  of  two  simulatic  t  ains  with  RVSM  rules  in  effect  It  must  be  noted  that 
there  will  be  no  increase  in  the  number  of  aircraft,  but  the  density  on  the  MTT  will  increase  due  to 
the  increased  number  of  available  altitudes 


The  second  experimental  condition  RVSM-CAT  will  be  conducted  on  the  third  and  fourth  days 
This  will  consist  of  simulation  runs  similar  to  basic  RVSM  until  iwo  pilots  report  greater  than 
moderate  CAT.  When  this  situation  occurs,  the  controller  is  required  to  comply  with  the  ATC  in¬ 
flight  contingency  procedures  for  RVSM  airspace  defined  in  the  NAT  Vertical  Separation 
Implementation  Group  (VSIG)  Meetings  1'  and  2**  The  purpose  of  these  simulation  runs  is  to 
examine  the  controller's  ability  to  comply  with  the  proposed  NAT  VSIG  procedures. 

3  3  CONFIGURATION  MANAGEMENT 

Configuration  management  will  ensure  repeatability  for  all  simulation  activities.  All  project  items 
used  in  connection  with  the  RVSM  project  will  be  documented  and  retained  in  a  library  within  the 
Human  Factors  Laboratory  building  at  the  FAA  Technic."!!  Center  Project  items  may  include 
documentation,  questionnaire  and  observer  fojms.  facility  tapes,  data  reduction  and  analysis 
(DR&A)  recording  tapes  and  listings,  and  simulation  run-time  logs.  Project  records  will  be 
accessible  for  review  by  project  management  and  customer/government  personnel 

Configuration  management  activities  will: 

a.  Assure  configuration  control  for  all  NSC  simulation  activities. 

b.  Ensure  compliance  with  simulation  procedures. 

c.  Document  problems  and  deficiencies  encountered  during  simulation. 

d.  Authenticate  and  assure  control  of  simulation  records 

e.  Maintain  and  control  all  inputs  and  outputs  (tape,  printout,  etc.)  during  the  entire 
simulation  period. 

f  Maintain  NSC  Documentation  Library. 


^Summary  of  Discussions  of  the  First  North  Atlantic  Vertical  Separatior.  implementation  Group  (NAT  VSlG/2)  of 
the  North  Atlantic  Systems  Planning  Group.  Paris,  18-22  Nov  1991 

^Summary  of  Discussions  of  the  Second  North  Atlantic  Vertical  Separation  Implementation  Group  (NAT  VSIG'2) 
of  the  North  Atlantic  Systems  Planning  Group.  Annapolis  30  March  to  3  April  1992 
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GLOSSARY 


Flight  Information  Region  (FIR)  -  Airspace  within  which  air  traffic  control  services  may  be 
provided. 

Minimum  Aircraft  System  Performance  Specification  tMASPS)  -Specification  defining  aircraft 
avionics  performance  standards  which  must  be  met  to  authorize  flight  in  the  MNPS 

Minimum  Navigation  Performance  Specifications  (MNPS)  -  A  specified  minimum  navigation 
performance  standard  whicii  aircraft  must  meet  in  order  to  operate  in  MNPS  designated  airspace 
In  addition,  aircraft  must  be  certified  by  their  state  of  registry  for  MNPS  operation.  The  objective 
of  MNPS  is  to  ensure  the  safe  separation  of  aircraft  and  to  derive  maximum  benefit,  generally 
through  reduced  separation  standard,  from  the  improvement  in  accuracy  of  navigation  equipment 
developed  in  recent  years. 

Minimum  Navigation  Performance  Specification  Airspace  IMNPSA)  -  A  portion  of  the  NAT 
airspace  between  FL275  and  FL400  extended  between  latitudes  27N  and  the  North  Pole,  bounded 
in  the  east  by  eastern  boundaries  of  control  areas  Santa  Maria  oceanic,  Shanwick  oceanic, 
Reykjavik,  and  in  the  west  by  the  western  boundaries  of  CTA  Gander  oceanic  and  the  western 
boundary  of  CTA  New  York  oceanic,  excluding  the  area  west  of  60W  and  south  of  38.40N. 

Western  Atlantic  Route  System  IWATRS)  -  Beginning  at  a  point  27  OON/77  OOW  direct  to 
20.00N/67.00W  direct  to  18.00N/62.00W  direct  to  18  00N/60.00W  direct  to  38.30N/60.00W 
direct  to  38.30N/69. 15W  thence  counterclockwise  along  the  New  York  oceanic  control 
area/flight  information  region  boundary  to  the  Miami  oceanic  control  area/flight  information 
region  boundary,  thence  southbound  along  the  Miami  oceanic  control  area/flight  information 
region  boundary  to  the  point  of  beginning. 

Sector  7 1  boundaries  -  Beginning  at  the  northeast  corner.  37.00N/40  OOW,  south  to 
27.00N/40.00W.  west  to  27.00N/60.00W.  north  to  3 1  .0ON/60.00W,  west  to  3  I  .OON/61 .33W. 
northwest  arc  on  the  Bermuda  boundary  to  34.41N/62.25W,  northeast  to  37.00N/60.00W,  east  to 
37.00N/55.00W,  east  to  37.00N/50.00W,  east  back  to  37.00N/40.00W. 

Sector  72  boundaries  -  Beginning  at  the  northeast  corner,  45  OON/40  OOW,  south  to 
37.00N/40.00W,  west  to  37.00N/60.00W,  southwest  to  34.41N/62.25W,  northwest  arc  on  the 
Bermuda  boundary  to  35  20N/65. 14W,  northwest  to  38.35N/69  02W,  northeast  to 
39.00N/67.00W,  north  to  41 ,52N/67.00W,  northeast  to  43.56N/60  01 W,  northeast  to 
44  40N/55.00W,  northeast  to  45.00N/53.00W,  east  back  to  45  OON/40  OOW. 
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ACRONYMS 


AC 

Advisory  Circular 

ACES 

Adaptation  Controlled  Environment  Subsystem 

ACJ 

Advisory  Circular.  Joint 

ADS 

Automatic  Dependent  Surveillance 

ANOVA 

Analysis  of  Variance 

ARIMA 

Auto-Regressive  Integrative  Moving  Average 

ARINC 

Aeronautical  Radio.  Inc  (a  suppon  contractor) 

ARTCC 

Air  Route  Traffic  Control  Center 

ATC 

Air  Traffic  Control 

ATWIT 

Air  Traffic  Workload  Input  Technique 

CAT 

Clear  Air  Turbulence 

CVSM 

Conventional  Vertical  Separation  Minima 

DART 

Data  Analysis  and  Reduction  Tool 

DR&A 

Data  Reduction  and  Analysis 

ESD 

Enhanced  Situation  Display 

FAA 

Federal  Aviation  Administration 

FDIO 

Flight  Data  Input/Output 

FIR 

Flight  Information  Region 

FL 

Flight  Level 

ft 

Feet;  foot 

GNSS 

Global  Navigation  Satellite  System 

HMU 

Height  Monitoring  Unit 

ICAO 

International  Civil  Aviation  Organization 

m 

Meter(s) 

MASPS 

Minimum  Aircraft  System  Performance  Specification 

MNPS 

Minimum  Navigation  Performance  Specification 

MTT 

Minimum  Time  Track 

NAT 

North  Atlantic 

NATSPG 

North  Atlantic  Systems  Planning  Group 

NSC 

National  Simulation  Capability 

ODAPS 

Oceanic  Display  and  Planning  System 

ODF 

Oceanic  Development  Facility 

OSRT 

Oceanic  System  Requirements  Team 

PVD 

Plan  View  Display 

RGCSP 

Review  of  the  General  Concept  of  Sepai  ation  Panel 

RVSM 

Reduced  Vertical  Separation  Minima 

SAR 

System  Analysis  and  Recording 

TLS 

Target  Level  of  Safety 

VHF 

Very  High  Frequency 

VSIG 

Vertical  Separation  Implementation  Group 

VSM 

Vertical  Separation  Minimum 

WATRS 

Western  Atlantic  Route  System 
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APPENDIX  A 

FULL  IMPLEMENTATION  SCHEDULE  FOR  RVSM 


November  1992 


January  1993 


February  1994 


June  1994 


October  1994 
January  1995 


April  1995 


January  1996 


a.  (LIM)  NAT  (RAN)  Meeting  endorsement  of  plans  and 
programs  for  the  implementation  of  RVSM;  and 
b  ATC  simulation  commences  (United  States). 

a.  Begin  field  evaluation  of  prototype  height  monitoring  unit 
(HMU);  and 

b.  ATC  simulation  commences  (Canada  and  Iceland). 

a.  Publish  Advisory  Circular/ Advisory  Circular  Joint  (AC/ACJ) 
and  distribute  to  NAT  user  states  for  consideration  and 

b.  Start  approval  of  aircraft/operators 

a.  Decide  on  system  performance  monitoring  technique  to  be 
used; 

b.  Acceptance  of  prototype  HMU  system;  and 

c.  HMU  contract  to  competitive  tender. 

a  Implement  state  and  central  data  base  of  approved  users. 

a.  Commence  building  operational  HMUs;  and 

b.  All  ATC  simulations  completed. 

a.  Target  of  50  per  cent  of  operators  approved  for  reduced 
VSM  operations; 

b.  Operational  HMUs  come  on-line; 

c.  ATC  procedures  and  transition  arrangements  finalized; 

d.  Start  system  verification  in  2,000  ft  VSM  environment; 

1 .  On  the  basis  of  data  collected,  ensure  that  aircraft  meet 
the  global  system  performance  specifications;  and 

2.  Ensure  that  the  system  meets  the  NAT  Region  TLS  of 
5.0  X  10'^  (second-half  1995). 

a)  Acceptance  (progressive)  of  HMUs;  and 

b)  All  ATC  facilities  and  arrangements  in  place. 

a.  Target  of  90  per  cent  of  operators  approved  for  RVSM 
operations, 

b.  Applicability  of  MASPS;  and 

c.  Applicability  of  the  NAT  SUPPs  relating  to  height-keeping 
performance. 


January  1997 


Januaiy  1998 


a  Confirm  that  at  least  90  per  cent  of  operators  approved  for 
RVSM  (the  remaining  10  per  cent  could  be  accommodated 
outside  MNPS  airspace); 

b.  Applicability  of  the  SUPPs  relating  to  the  implementation  of 
1.000  ft  VSM; 

c.  Implementation  of  the  transition  areas; 

d.  Start  of  operational  trials  in  1,000  ft  VSM  environment;  and 
e  Aircraft  and  operators  not  approved  for  operations  in  MNPS 
airspace  (including  MASPS)  will  be  excluded. 

Full  implementation 
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APPENDIX  B 

SIMULATION  CONTROLLER  FORMS/QUESTIONNAIRES 


NSC  SIMULATION:  CONTROLLER  FORM  I. 
INSTRUCTIONS: 


1.  This  form  is  to  be  completed  by  all  controllers  prior  to  participation  in  the  NSC  Phase  One 
Simulation  acdvides.  The  form  consists  of  requests  for  general  background  infcvmadon  and  an 
inidal  (baseline)  judgment  regarding  oceanic  control  praedees. 

2.  Controllers  should  be  advised  that  their  names  will  not  be  listed  or  appear  in  any  of  the  NSCs 
data  records  to  insure  anonymity  and  to  encourage  unbiased  repordng.  Findings  will  be  reported 
as  group  data  and  generically,  as  Controller  A,  B,  C,  etc.  In  order  to  facilitate  data  analysis,  the 
experimenters  wUl  use  the  last  four  digits  of  a  controller's  social  security  number  tc  ,  .date  various 
data  records  belonging  to  a  pardcular  simuiadon  subject. 
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NSC  SIMULATION 


AC 


rn 


am  m 


years 


months 


Last  four  digits  of  your  Social  Security  Number:  _ 

Age:  _ 

How  many  years  of  experience  as  a  controller  iFPL)  ?  _ years _ me 

How  many  years  of  experience  as  an  Oceanic  FPL?  _ years  _  moa.tl 

How  much  time  as  an  oceanic  controller  in  the  North  Atlantic  Region?  _ 

Name  the  last  three  facilities  at  which  you  have  worked  starting  with  your  current 
assignment: 

(1) _ (2) _  (3) _ 


ERAL  PROCEDURES  &  PRACTICES  INFORMATION 


1.  If  you  had  an  opportunity  to  change  three  current  elements  in  the  oceanic  area 
(practices,  procedures,  equipment,  etc.),  what  would  they  be? 


months 


2.  Based  on  your  experience  with  high  traffic  and  high  workload  in  oceanic  operations; 

a)  What  are  some  of  the  things  that  can  occur  that  could  cause  you  to  have  significant 
difnculties  in  maintaining  an  orderly  and  expedititous  traffic  flow  ? 


b)  Which  of  these  events  tend  to  occur  most  frequently 


c)  Which  of  these  events  tend  to  most  likely  cause  additional  problems' 


3.  Please  check  all  items  below  that  you  feel  contribute  significantly  to  high  levels  of 
workload  in  the  current  oceanic  ATC  system: 


□  Printer  Speed  ODAPS _  ARINC 

□  ODAPS  Update  Rate 

□  ARINC  Communication  Delays 
G  Phone  System 
G  Special  Pilot  Requests 
G  Other 


G  Coordination  with  Foreign  Facilities 
G  Coordination  with  Fellow  Controllers 
G  Oceanic  Track  System 
G  Random  Routes 

G  Aircraft  Performance  Characteristics/Mix 
G  Other 


Comments  on  any  of  the  above  checked  items: 


NSC  SIMULATION:  CONTROLLER  FORM  II 
INSTRUCTIONS: 


1.  This  form  is  to  be  completed  by  all  controllers  after  each  completed  simulation  run  in  the 
NSC  Phase  One  Simulation  activities.  The  form  consists  of  requests  for  information 
regarding  overall  experiences  and  judgments  about  the  just  completed  simulation  run. 

2.  The  RATING  NUMBERS  to  be  used  for  item  #4  are: 

1  =  Remarkably  good 

2  =  Moderately  good 

3  =  So-so 

4  =  Not  very  good 

5  =  Unusually  poor 

3.  Page  2  of  the  Form  consists  of  two  different  versions:  Version  1  is  to  be  filled  out  after 
CVSM  (baseline)  simulation  runs.  Version  2  is  to  be  filled  out  after  simulations  with 
RVSM  conditions.  The  difference  between  these  two  versions  is  that  version  2  elicits 
additional,  RVSM-specific  information. 
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NSC  SIMULATION:  CONTROLLER  FORM  11 
POST  SIMULATION  RUN  QUESTIONNAIRE 


Simulation  Condition:  [D  CVSM  O  RVSM  RVSM-  MTT  O  RVSM-  CAT 
Your  SS#-(4  last  digits)  _ Date: _ Simulation  Run  No: _ 


1.  Please  estimate  your  overall 
one)? 


during  the  last  simulation  (circle 


2.  In  terms  of  REALISM,  how  real  was  the  PHYSICAL  SIMULATION 
ENVIRONMENT  for  you  (circle  one)?  _ 


1  23456789  10 


VERY  LOW 


MODERATE 


3.  In  terms  of 
TRAFFIC  for  you  (circle  one) 


12  3  4 

I - 1 - 1 - 

VERY  LOW 


VERY  HIGH 


how  real  was  the  SIMULATED 


MODERATE 


VERY  HIGH 


4.  Please  judae  vour  OWN  PERFORMANCE  over  the  most  recent  simulation  run. 
Using  the  RATING  NUMBERS  from  the  instruction  sheet,  for  each  position 
worked,  insert  the  appropriate  rating  into  each  box  next  to  the  following  factors: 

ODAPS-POSITION 

D-POSITION 

^  Proper  Coordination 

O  Promptness  of  Actions 
(3  Situation  Awareness  Maintenance 
d  Communication  Management 

Proper  Message  Construction 

O  Computer  Entry  Management 

C3  Other 

CD  Proper  Coordination 

O  Promptness  of  Actions 

Situation  Awareness  Maintenance 
Communication  Management 

O  Proper  Clearances  Issued 

CD  Flight  Strip  Management 

O  Maintenance  of  Separation 

5.  What  was  most  difficult  for  you  to  accomplish  during  the  simulation  ? 
a)  (at  the)  ODAPS  position  b)  (at  the)  D-Position 


CVSM  (BASELINE)  CONDITION  CONTROLLER  FORM  U 

continued 


6.  If  you  could  change  something  about  the  last  simulation  run  (anything  at  all 
about  the  traffic  scenario,  aircraft,  procedures,  etc.,  etc.)  ,  what  would  it  be  ? 


7.  Did  you  change  your  usual  control  and  work  strategies  in  any  way  in  order  to 
work  the  traffic  in  the  last  simulation?  If  so,  how?  What  did  you  do  differently? 


8.  Based  upon  your  experience  with  the  traffic  load  during  the  last  iimulation 
run,  what  procedures  would  have  to  be  changed  and/or  implemented  in  order  for 
you  to  continue  to  be  comfortable  about  working  this  same  traffic  but  under 
reduced  vertical  separation  minima  (RVSM)? 


9.  What  was  your  primary  safety  concern  considering  traffic,  events,  and 
procedures  in  the  last  simulation  run? 


RVSM  CONDITIONS  CONTROLLER  FORM  II 

continued 


6.  If  you  could  change  something  about  the  last  simulation  run  (anything  at  all 
about  the  traffic  scenario,  aircraft,  procedures,  etc.,  etc.)  ,  what  would  it  be  ? 


7.  Did  you  change  your  usual  control  and  work  strategies  in  any  way  in  order  to 
work  the  traffic  with  RVSM?  If  so,  how?  What  did  you  do  differently? 


8.  Based  upon  your  experience  with  RVSM  in  the  last  simulation  run,  what 
procedures  would  have  to  be  changed  and/or  implemented  in  order  for  you  to 
continue  to  be  comfortable  about  working  this  traffic  ? 


9.  What  was  your  primary  safety  concern  considering  traffic,  events,  and 
procedures  in  the  last  simulation  run? 


NSC  SIMULATION:  OBSERVER  FORM  1. 
INSTRUCTIONS: 


1.  This  fonn  is  to  be  completed  by  all  technical  observers  prior  to  participation  in  the  NSC  Phase 
One  Simulation  activities.  The  form  consists  of  requests  for  general  background  information  and 
an  initial  (baseline)  judgment  regarding  oceanic  control  practices. 

2.  Observers  should  be  advised  that  their  names  will  not  be  listed  or  appear  in  any  of  the  NSCs 
data  records  to  insure  anonymity  and  to  encourage  unbiased  reporting.  Findings  will  be  rqxirted 
as  group  data  and  genetically,  as  Controller/Observer  1, 2,  3,  etc.  In  order  to  facilitate  data 
analysis,  the  experimenters  will  use  the  last  four  digits  of  an  observer's  social  security  number  to 
collate  various  data  records  belonging  to  a  particular  simulation  subject 
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NSC  SIMULATION: 


GENERAL  BaU 


Date: 


r<H;TITIKriMlI?raiT:i;^E¥¥filM 


Last  four  digits  of  your  Social  Security  Number:  _ 

Age:  _ 

How  many  years  of  experience  as  a  controller  (FPL)  ?  _ years 

How  many  years  of  experience  as  an  Oceanic  FPL  ?  _ years  _ 

How  much  time  as  an  oceanic  controller  in  the  North  Atlantic  Region? 

How  many  years/months  since  you  have  controlled  oceanic  traffic?  _ 

How  many  years/months  as  a  trainer/developer  in  an  ATC  facility  ?  _ 


months 


months 


WJivTancnaaianfTOiRTaigi! 


PRACTICES  INFORMATION 


1.  If  you  had  an  opportunity  to  change  three  current  elements  in  the  oceanic  area 
(practices,  procedures,  equipment,  etc.),  what  would  they  be? 


2.  Based  on  your  experience  with  high  traffic  and  high  workload  in  oceanic  operations; 

a)  What  are  some  of  the  things  ihat  can  occur  that  could  cause  a  controller  to  have 
significant  difficulties  in  maintaining  an  orderly  and  expedititous  traffic  flow  ? 


b)  Which  of  these  events  tend  to  occur  most  frequently 


c)  Which  of  these  events  would  most  likely  tend  to  cause  additional  problems 


3.  Please  check  all  items  below  that  you  feel  contribute  significantly  to  high  levels  of 
workload  in  the  current  oceanic  ATC  system: 


□  Printer  Speed  ODAPS _ ARINC. 

□  ODAPS  Update  Rate 

□  ARINC  Communication  Delays 

□  Phone  System 

□  Special  Pilot  Requests 

□  Other 


□  Coordination  with  Foreign  Facilities 
O  Coordination  with  Fellow  Controllers 
O  Oceanic  Track  System 

O  Random  Routes 

□  Aircraft  Performance  Characteristics/Mix 
O  Other 


Comments  on  any  of  the  above  checked  items: 


NSC  SIMULATION:  TECHNICAL  OBSERVER  ’’QUICK”  FORM. 

INSTRUCTIONS: 

1.  This  form  is  to  be  completed  by  trained  technical  observers  during  each  simulation  run. 

2.  Observations  and  responses  will  be  recorded  in  15  minute  intervals.  A  new  (identical) 
form  will  be  used  for  each  15  minute  time  period.  As  much  information  as  possible  will  be 
preprinted  on  the  forms  (example,  the  interval  will  be  specified  as  T+15,  T+30,  T+45, 
etc.). 

3.  Observations  will  be  made  on  this  form  primarily  for  the  "D-Position"  controller. 
However,  both  controllers  will  be  prompted  for  their  workload  rating  at  15  minute  intervals 
and  the  observer  will  make  "quick  evaluations  ’  about  performance  on  both  positions. 

4.  The  rating  numbers  for  WORKLOAD  are  on  a  scale  from  1  to  10  with 

1  =  very  low  workload 
5  =  moderate  workload 
10  =  very  high  workload 

5.  The  rating  numbers  for  the  rest  of  the  QUICK  EVALUATIONS  are 

1  =  Remarkably  good 

2  =  Moderately  good 

3  =  So-so 

4  =  Not  very  good 

5  =  Unusually  poor 
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INTERVAL:  _ 

Simulation  Run  No. _ 

Date: _  Sector  Number: 


TIME: 


Date: _  Sector  Number: _  Observer  ID _ 


Prompt  tor  workload  rating  trom  the  controllers  every  15  minutes! 
WORKLOAD  RATING  -  "D"ContrDller  :  123456789  10 

WORKLOAD  RATING  -  "ODAPS"  Controller  :1234S6789  10 


RESTRICTIONS  ISSUED  (hashmarks): 


□  SPEED 


□  TIME 


□  ALTITUDE 


□  ROUTING 


ARINC  COMMUNICATIONS  GROUPINGS  (hashmarks): 


OPERATIONAL  ERROR  OBSERVED: 


TIME: 

□  Loss  of  Separation 

Briefly  describe  what  happened: 

□  Other 

UICK  OBSERVER  EVALUATIONS  (circle  one) 


"ODAPS" 

CONTROLLER  I  "D"  CONTROLLER 

12  3 

4 

WORKLOAD  RATING 

56789  10  1  1234567 

8 

9  10 

1 

2 

COORDINATION  &  COMMUNICATION 

3  4  5  1  1  2  3 

4 

5 

1 

2 

TRAFFIC  MANAGEMENT  &  CONTROL  JUDGMENT 

3  4  5  1  1  2  3  4 

5 

1 

2 

FLIGHT  STRIP  MANAGEMENT 

3  4  5  1  1  2  3 

4 

5 
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NSC  SIMULATION:  OBSERVER  FORM  IL 
INSTRUCTIONS: 

1.  This  fonn  is  to  be  completed  by  all  observers  after  each  completed  simulation  run  in  the 
NSC  Phase  One  Simulation  activities.  The  form  consists  of  requests  for  information 
regarding  overall  experiences  and  judgments  about  the  just  completed  simulation  run. 

2. The  RATING  NUMBERS  to  be  used  for  item  #4  are: 

1  =  Remarkably  good 

2  =  Moderately  good 

3  =  So-so 

4  =  Not  very  good 

5  =  Unusually  poor 

3.  Page  6  of  the  Form  consists  of  two  different  versions:  Version  1  is  to  be  filled  out  after 
CVSM  baseline  simulation  runs.  Version  2  is  to  be  filled  out  after  simulations  with  RVSM. 
The  difference  between  these  two  versions  is  that  version  2  elicits  additional,  RVSM- 
specific  information. 
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NSC  SIMULATION:  OBSERVER  FORM  II. 
POST  SIMULATION  RUN  QUESTIONNAIRE 


Simulation  Condition:  □  CVSM  □  RVSM  □  RVSM-MTT  □  RVSM-CAT 
Observer  ID#  _ Date: _ Simulation  Run  No.: _ 


la.  For  each  controller  please  estimate  the  overall  "D"  position  WORKLOAD 
during  the  last  simulation  (circle  one): 


Controller  SS#  (four  digits). 


1  2 

3 

4  5  6 

7 

8 

9  10 

VERYLOW 

1 

III 

MODERATE 

1 

1 

1  1 

VERY  HIGH 

Controller  SS#  (four  digits). 


1  2 

3 

4  5  6 

7 

8 

9  10 

I  1 

VERYLOW 

1 

111 

MODERATE 

1 

1 

1  1 

VERY  HIGH 

Controller  SS#  (four  digits). 


1  2 

3 

4  5  6 

7 

8 

9  10 

1  1 

VERYLOW 

1 

1  1  1 

MODERATE 

1 

1 

VERY  HIGH 

Controller  SS#  (four  digits). 


1  2 

3 

4  5  6 

7 

8 

9  10 

VERYLOW 

1 

11  1 

MODERATE 

1 

1 

1  1 

VERY  HIGH 
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NSC  SIMULATION: 


continued 


lb.  For  each  controller  please  estimate  the  overall  ODAPS  position 
during  the  last  simulation  (circle  one): 

Controller  SS#  (four  digits) _ 


VERY  LOW 


1  2 


VERY  LOW 


VERYLOW 


1  2 


MODERATE 


MOCEFATE 


MODERATE 


VERY  HIGH 


Controller  SS#  (four  digits) _ 


3456789  10 


VERY  HIGH 


Controller  SS#  (four  digits) _ 


3456789  10 


VERY  HIGH 


Controller  SS#  (four  digits) _ 


3456789  10 


VERYLOW 


MODERATE 


VERY  HIGH 


2.  In  your  opinion,  in  terms  of  REALISM,  how  real  was  the  PHYSICAL 
SIMULATION  ENVIRONMENT  for  the  controllers  (circle  one)? 


VERYLOW 


MODERATE 


VERY  HIGH 


3.  In  your  opinion,  in  terms  of  FUNCTIONAL  REALISM,  how  real  was  the 
SIMULATED  TRAFFIC  for  the  controllers  (circle  one)? 


VERYLOW 


MODERATE 


VERY  HIGH 
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NSC  SIMULATION:  OBSERVER  FORM  H. 


continued 


4.  Please  judge  OVERALL  CONTROLLER  PERFORMANCE  during  the  most 
recent  simulation  run. 

Iking  thf>  RATINO  NUMBERS  from  the  instruction  sheet,  insert  the  aoDrODriate 
rating  number  for  each  of  the  oositions  observed  into  the  box  next  to  each  of  the 
following  factors: 

Controller  SS#  (4  dig 

ODAPS-POSITION 

itsi 

D-POSITION 

Proper  Coordination 

d  Promptness  of  Actions 

Gi  Situation  Awareness  Maintenance 

G  Communication  Management 

G  Proper  Message  Construction 

G  Computer  Entry  Management 

G  Other 

G  Proper  Coordination 

G  Promptness  of  Actions 

G  Situation  Awareness  Maintenance 
G  Communication  Management 

G  Proper  Clearances  Issued 

G  Flight  Strip  Management 

G  Maintenance  of  Separation  | 

-  -r 

Controller  SS#  (4  dig 

ODAPS-POSITION 

its) 

D-POSITION  1 

G  Proper  Coordination 

G  Promptness  of  Actions 

G  Situation  Awareness  Maintenance 

G  Communication  Management 

G  Proper  Message  Construction 

G  Computer  Entry  Management 

G  Other 

G  Proper  Coordination 

G  Promptness  of  Actions 

G  Situation  Awareness  Maintenance 
G  Communication  Management 

LJ  Proper  Clearances  Issued 

G  Flight  Strip  Management 

G  Maintenance  of  Separation 
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NSC  SIMULATION;  OBSER> 
continued 


Controller  SS#  (4  digits). 
ODAPS-POSITION  I 


D-POSITION 


O  Proper  Coordination 

Proper  Coordination 

C7  Promptness  of  Actions 

Promptness  of  Actions 

Situation  Awareness  Maintenance 

d  Situation  Awareness  Maintenance 

Communication  Management 

d  Communication  Management 

d  Proper  Message  Construction 

C3  Proper  Clearances  Issued 

O  Computer  Entry  Management 

d  Flight  Strip  Management 

O  Other 

d  Maintenance  of  Separation 

Controller  SS#  (4  digits). 


ODAPS-POSITION 


D-POSITION 


d  Proper  Coordination 

d  Proper  Coordination 

d  Promptness  of  Actions 

d  Promptness  of  Actions 

d  Situation  Awareness  Maintenance 

d  Situation  Awareness  Maintenance 

d  Communication  Management 

d  Communication  Management 

d  Proper  Message  Construction 

d  Proper  Clearances  Issued 

d  Computer  Entry  Management 

d  Flight  Strip  Management 

d  Other 

d  Maintenance  of  Separation 

5.  What  was  most  difficult  for  controllers  to  accomplish  during  this  simulation  ? 

a)  (at  the)  ODAPS  position 

b)  (at  the)  D-Position 
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CVSM  (BASELINE)  CONDITION  OBSERVER  FORM  II 

continued 


6.  If  you  could  change  something  about  the  last  simulation  run  (anything  at  all 
about  the  traffic  scenario,  aircraft,  procedures,  etc.,  etc.)  ,  nhat  would  it  be  ? 


7.  Did  the  controllers  change  their  usual  control  and  work  strategies  in  any  way 
in  order  to  work  the  traffic  in  the  last  simulation?  If  so,  how?  What  did  they  do 
differently? 


8.  Based  upon  your  observations  with  the  traffic  load  during  the  last  simulation 
run,  what  procedures  would  have  to  be  changed  and/or  implemented  in  order  for 
the  controllers  to  be  comfortable  about  working  this  same  traffic  but  under 
reduced  vertical  separation  minima  (RVSM)? 


9.  What  was  your  primary  safety  concern  considering  traffic,  events,  and 
procedures  in  the  last  simulation  run? 
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RVSM  CONDITIONS  OBSERVER  FORM  U 
continued 


6.  If  you  could  change  something  about  the  last  simulation  run  (anything  at  all 
about  the  traffic  scenario,  aircraft,  procedures,  etc.,  etc.)  ,  what  would  it  be  ? 


7.  Did  the  controllers  change  their  usual  control  and  work  strategies  in  any  way 
in  order  to  work  the  traffic  with  RVSM?  If  so,  how?  What  did  they  do 
differently? 


8.  Based  upon  your  observations  with  RVSM  in  the  last  simulation  run,  what 
procedures  would  have  be  changed  and/or  implemented  in  order  for  the 
controllers  to  continue  to  be  comfortable  about  working  this  traffic  ? 


9.  What  was  your  primary  safety  concern  considering  traffic,  events,  and 
procedures  in  the  last  simulation  run? 
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